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Abstract 



We approximate a given rational spectral density by one that is consistent with prescribed second- 
order statistics. Such an approximation is obtained by minimizing a suitable distance from the given 
spectrum and under the constraints corresponding to imposing the given second-order statistics. Here, 
we consider the Alpha divergence family as a distance measure. We show that the corresponding 
approximation problem leads to a family of rational solutions. Secondly, such a family contains the 
solution which generalizes the KuUback-Leibler solution proposed by Georgiou and Lindquist in 2003. 
Finally, numerical simulations suggest that this family contains solutions close to the non-rational 
solution given by the principle of minimum discrimination information. 

1 Introduction 

This paper deals with the rational approximation of power spectra of stationary stochastic processes. 
More precisely, we consider the following situation. Let y = {y^', A: e Z} be a zero mean, M- valued, 
purely non-deterministic, full-rank, stationary process with unknown spectral density defined on 

the unit circle T. A rational prior spectral density ^' € S-(_(T), which represents the a priori information 
on y, is available. Here, S+(T) denotes the family of bounded and coercive M- valued spectral density 
functions on T. Then, some second-order statistics of y are observed. These are encoded in the output 
covariance, denoted by S, of a rational filters bank G{z) = {zI—A)~^B driven by y. The filter parameters 
are chosen in such a way that A e M"^" is a stability matrix, B G M", and {A,B) is a reachable pair 
with n > 1. Thus the output process, denoted hy x = {xi^; k e Z}, is a stationary process, and 
E = K[xkx'^] = J G^IG* is a positive definite matrix. Here, integration takes places on [— tt, tt) with 
respect to the normalized Lebesgue measure d'i9/27r. The rational prior ^ is typically inconsistent with 
E, i.e. E 7^ J G'i'G* . Hence, our task consists in finding a rational spectral density $ e §-|_(T) which is 
as close as possible to in some suitable distance among spectral densities and such that 



In other words we reconcile the inconsistence among E and ^ by approximating the spectrum \1/ by a 
rational spectrum $ compatible with E. This problem is motivated by THREE-like spectral estimation 
paradigms [H [THl UHl [HI HOj wherein $ represents an estimate of the unknown spectral density 
compatible with the given second-order statistics of Q and as close as possible to the a priori information 
given. 

The role of the filters bank G(z) consists in providing the interpolation conditions for the solution to 
the spectrum approximation problem. More specifically, by choosing the filters bank poles appropriately, 
it is possible to give preference to selected frequency bands of $ and allow more accurate reconstruction 
of in these particular frequency bands, @]. 

Concerning the exploitable distance measures in the above problem, we have many options. In 
the KuUback-Leibler divergence among spectral densities having the same zeroth moment is considered 
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The found rational solutfon $ is the spectral density satisfying ([T]) which minimizes 5KLo(^||*i')- Note 
that in statistics, information theory and communications, the Kullback-Leihler divergence is typically 
minimized with respect to the first argument, [HI [7], according to the Principle of minimum discrimination 
information, MinxEnt, 17J. ft states that, given new information, a new distribution $ should be chosen 
in such a way as to minimize 5klo(*1'||^)- In |16) . the unusual choice to minimize with respect to the 
second argument is dictated by the fact that the optimization of 5klo(^||^) leads to a non-rational 
"exponential- type" solution, [15]. Then, in [10] and [8], the rational solutions corresponding to the 
Hellinger distance and the Itakura-Saito distance, respectively, are considered. Finally, in |20j the Beta 
divergence family is considered. The latter smoothly connects the Kullback-Leihler divergence with the 
Itakura-Saito distance. Making additional assumptions on ^ besides the rationality, it is possible to 
prove that the Beta divergence leads to a family of rational solutions. Finally, it is worth to note that 
the solutions considered in [T0l[8l|20] also hold for the multichannel case, i.e. ?/ is a multivariate process. 

The aim of this paper is to solve the previous spectrum approximation problem by employing the 
Alpha divergence family which smoothly connects the Kullback-Leihler divergence 

5kl($i||*2) j $1 fog - $1 -t- $2 (3) 

with its reverse 5KL($2||'f i), passing through the Hellinger distance. Firstly we will generalize the 
solution in [TFj to spectral densities with different zeroth moment. Then, we will see that it is possible to 
characterize a family of rational solutions to the problem without making additional assumptions on ^ . 
The limit of this family of solutions has the same "exponential-type" structure as the MinxEnt solution. 
Furthermore, simulations suggest that the limit above converges to the non-rational solution given by 
'5kl(^'||^')- Thus, the spectrum approximation problem based on the Alpha divergence family also seems 
to provide rational solutions, for a suitable choice of the a parameter, closer to the MinxEnt solution. 



2 Spectrum approximation problem and feasibility conditions 

The spectrum approximation problem we are dealing with minimizes a suitable distance measure, denoted 
by ^f), over the set 

:= ($ e §+(T) I /g$G* (4) 



where ^ £ S+(T) is rational, and G{z) = {zl — A)~^B has the same properties specified in the Introduc- 
tion. Both and G(z) are given. The output covariance E is estimated by a given finite-length sequence 
yi .. .yjy, extracted from a realization of y, see [mUT]. Thus, the first problem concerns the feasibility 
of the spectrum approximation problem, i.e. when the set Tx; is non-empty for a given E. To deal with 
this issue, we first introduce some notation: Q„ C M"^" denotes the n(n -I- l)/2-dimensional real vector 
space of n-dimensional symmetric matrices and Q„,+ denotes the corresponding cone of positive definite 
matrices. We denote as V(§+(T)) the linear space generated by §+(T). Finally, we introduce the linear 
operator 

r : V(§+(T))^Q„ 

/ G$G*. (5) 



In [13], it was shown that a matrix P e Q„ belongs to the range of F, denoted by Range F, if and only 
if there exists e M^^" such that 

P - APA^ = BH + H'^B'^. (6) 

It turns out that the spectrum approximation problem is feasible if and only if E G Range F n Qn,+ , see 
|14l 110) . Once we have E in such a way that the spectrum approximation problem is feasible, we can 
replace G with G = E'sG and {A,B) with (A = T.^ i AT,i ,B = T,' ^ B) . Thus, constraint ^ may be 
rewritten as 

G$G* = /. (7) 
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Accordingly, from now on we assume that our spectrum approximation problem is feasible and we consider 
the following equivalent formulation: Given a rational prior "if E §+(T) and G{z) such that / e Range F, 
minimize S{^, 5*) over the (non-empty) set 

|$e§+(T)| Jg^G*^!^ (8) 

In the following section we will introduce the last element characterizing our problem: The distance 
measure 5($, ^'). 



3 Spectrum approximation problem with the Alpha divergence 

The Alpha divergence family, [Hll], among two spectral densities $1, $2 G S+fT) is defined as 

5„($i|l$2) := / ^ ,, $?$2~° - + -$2 (9) 

J a(a — 1) a — 1 a 

where a is a real parameter. For a = and a = 1, it is defined by continuity 

lim5„($i||$2) =5kl($2||$i), lim5„($i||$2) =5kl($i||$2). (10) 

a— >-0 a— >-l 

Thus, the Alpha divergence is a continuous function of real variable a in the whole range including 
singularities and it smoothly connects iSKL(*i'i||^2) with its reverse iSKL('52||*i'i)- Moreover, Sa is strictly 
convex with respect to both $1 and $2- Note that, for a = ^ we obtain, up to a constant factor, the 
Hellinger distance 

5h(*i,*2) :=y"(v^- V^)' (11) 
and for a = 2 we obtain the Pearson Chi-square distance 

1 /■ ($1 - $9)^ 

5p(<i>i||$2) 2 y \, ■ (12) 

Since a G M, we choose the parametrization a ^ — ^ + 1 with i/ G N such that 1 < ly < 00 and we 
consider the following spectrum approximation problem. 

problem 3.1 Given a rational prior 'i' E §+(ir) and G{z) = (zl — A)~^B such that I E Range F, 

minimize iS,y(<i>||^') over the set 



$e§+(T) I J G$G* 



(13) 



5kl(*||$), 1^=1 
5,($||*) := <j /^$^*^ l<iy<oo (14) 
5kl($||^'), iy^<x 



where 



In the next sections we show that Problem |3.1[ which is a constrained convex optimization problem, 
admits a unique solution once fixed G N such that I < v < 00. This task is accomplished by exploiting 
the duality theory, [3]. We not only show that the dual problem admits solution but we even prove that 
there exists a suitable subset of the dual functional domain which contains a unique optimal Lagrange 
multiplier for Problem |3.1[ Accordingly, it is possible to employ the efficient Newton-type algorithm 
presented in [H] for computing the numerical solution of the problem. Moreover, thanks to the chosen 
parametrization i/, we will show that the solution is rational with a prescribed maximum degree when 



v < 00. Thus, Problem 3.1 with v < 00 can be viewed as a Nevanlinna-Pick interpolation problem with 
bounded degree, p| [12]. 

The analysis will be divided in the following three cases: j/ = l,l<z^<oo and v — 00. 
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4 Case u = 1 

The corresponding Lagrangian is 

L($,A) := 5kl(*||$) + y <^y"G$G* -/,A^ 

= J [-5- log(*-i$) + $ + G* AG$] - tr(A) (15) 

where we exploited the fact that / "if plays no role in the optimization task. The Lagrange multiplier 
A € Qn can be uniquely decomposed as A = Ap + Aj^ where Ap £ Range F and A_l G [Range T]-^. Since 
A_L is such that G* (eJ'')A_LG(eJ'') = and tr(A_L) = (A_l,/) = 0, see [H Section III], it docs not affect 
the Lagrangian, i.e. L($, A) = L($, Ar). Accordingly we can impose from now on that A G Range F. 
Consider now the unconstrained minimization problem 

min{L($, A) I $ e §+(¥)}. (16) 

Since 5kl(^||*1') is strictly convex with respect to $ e S+(T), then L{-, A) is strictly convex with respect 
to $ e S-i-(T). Accordingly, the unique minimum point of L(-, A) is given by annihilating its first variation 
in each direction £ Loo(Tr): 

SL{^,A;5^) = y"[(-*$"^ + l + G*AG)(5$] . (17) 

Note that, + 1 + G*AG e Loo(T). Hence, ^ is zero V(5$ G Loo(T) if and only if 

_ + 1 + G*AG = 0. (18) 

Therefore, the unique minimmn point of L{-,A) has the form 

As E §_|_(T), the set of admissible Lagrange multipliers is 

■■= {A e Q„ I 1 + G*AG > on T} n Range F. (20) 
Since $(A) is the unique minimum point of L(-, A), we get 

L($(A),A) < i($. A), V$ e §+(T) s.t. $ 7^ $(A). (21) 



Hence, if we produce A° e C satisfying constraint in (13 1, inequality (21) implies 

5kl(*||*(A°)) < 5kl(*||*), V$ e §+(T) s.t. J G$G* = E (22) 
and equality holds if and only if $ = $(A°). Accordingly, such a $(A°) is the unique solution to Problem 



3.1 Note that, $(A) is rational because ^'(z) is a rational function and G(z) is a rational matrix function. 



Furthermore, it is possible to characterize an upper bound on its degree: 

deg[$(A)] < deg[^'] + 2n. (23) 

The following step consists in showing the existence of such a A° by exploiting the duality theory. The 
dual problem consists in maximizing the functional 

infi($,A) = L($(A),A) = y"*log(l + G*AG) + «'-tr(A) (24) 

which is equivalent to minimize the following functional hereafter referred to as dual functional: 

J(A) = /-5'log(l + G*AG)+tr(A). (25) 
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Theorem 4.1 The dual functional J belongs to C'^{C^) and it is strictly convex over . 
Proof. The first variation of J(A) in direction SAi E Qn is 

(5J(A; SAi) = - y" *(1 + G*AG)-^G*dAiG + tr((5Ai). (26) 

The linear form VJa(-) ■— dJ{A; •) is the gradient of J at A. In order to prove that J{A) e C^{£^) we 
have to show that SJ{A;6Ai), for any fixed <5Ai, is continuous in A. To this aim, consider a sequence 
Mr, e Range T such that M„ ^- and define Qn{z) = (1 + G{z)* NG{z))-'^ with N e Q„. By Lemma 
5.2 in [19], Qk+m„ converges uniformly to Qa. Thus, applying the bounded convergence theorem, we 
obtain 

lim / G^Qk+m^G* = f G^QaG*. (27) 



Accordingly, 5J{A;SA) is continuous in A, i.e. J belongs to C^{C^). The second variation in direction 
Mi,(5A2 G Q„ is 

5V(A;(5Ai,(5A2) = j -^{l + G* AG^^G* SA^GG* 5A2G. (28) 

The bilinear form T-Ia{-, •) = (5^ J(A; •, •) is the Hessian of J at A. The continuity of S^J can be established 
by using the previous argumentation. Finally, it remains to be shown that J is strictly convex on the 
open set £^ . Since J € C^{£^), it is sufficient to show that Ha((5A, (5A) > for each 6A G Range T 
and equality holds if and only if 5 A = 0. Since the integrand in (28 1 is a nonnegative function when 
6A1 = M2, we have •Ha(M,<JA) > 0. If -Ha(^A,(5A) = 0, then G*SAG = namely 5A e [Range r]-^, 
see [ini Section III]. Since SA G Range T, it follows that i5A = 0. In conclusion, the Hessian is positive 
definite and the dual functional is strictly convex on ■ 

In view of Theorem 4.1 the dual problem nun { J(A) \ A £ C^^ admits at most one solution A°. Since 



is an open set, such a A° (if it does exist) annihilates the first directional derivative (261 for each 
SAe Qr. 



^-yt^ d + G-AoG) ^^^^)-"' (29) 

or, equivalently, 

^ = jG^^^^f-lGmiG*. (30) 



This means that $(A°) G §+(Tr) satisfies constraint in (13 1 and ^(A") is therefore the unique solution to 
Problem 13.11 

Although the existence question is quite delicate, since set is open and unbounded, we now show 
that such a A° minimizing J over does exist. 

Theorem 4.2 The dual functional J has a unique minimum point in £^ . 



Proof. In view of Theorem l4.ll it is sufficient to show that J takes a minimum value over . Consider 
the closure of 

£r = {A e Ranger I 1 + G*AG > onT} (31) 
and define the sequence of functions on 

J"(A) := y-*log (^1 + G*AG+ +tr(A). (32) 

By Lemma 1, Lemma 2 and Lemma 3 in [5] we conclude that: 

• the pointwise limit J°°(A) — lim„_>.oo J" (A) exists and is a lower semi-continuous, convex function 
on with values in the extended reals 

• J°° is bounded below on £^ 



J°°(A) = J(A) on £ 



r 
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• J°°{A) is finite on which is the complement set of B := {A e dC^ \ 1 + G*AG = 0} 

• Hm||A||^oo -'"(A) oo. 

Thus, J is inf-compact over and it admits a minimum point in A° G . Clearly, J(A) = oo for A Cz B. 
Let A e B", thus J(A) is finite. Since / G then A + e{I - A) e £J' with e e [0, 1]. The one-side 
directional derivative is 



lim 

e\,0 



J(A + e(/- A))- J(A) 



r'*r' -I- 1 

/ _ + tr(/ - A) 

1 + G*AG ^ ' 



-oo. 



(33) 



Thus, A cannot be a minimum point. We conclude that A° E . ■ 
Now, we point out the differences among our solution with v — 1 and the solution given in [16; . The 
latter is obtained by minimizing iSKLo(^ll'f) in and the corresponding optimal form is 



$KLo(A) 



G*AG' 



(34) 



As noticed in the Introduction, 5KLo('I'||*i^) is a distance measure among spectral densities having the 
same zeroth moment. However, when the matrix A is singular the zeroth moment of $ is fixed by 



constraint (13) 



where v E M" is such that v*A~0, see [ 
5kl(*||*) -5klo(*II$) 



Hence 



V V 

\v*B\\ 



$-5klo(*I1$) 



(35) 



(36) 



and the minimization of iSkl(^||"J') is equivalent to the minimization of iSklo(^II^)- We conclude, without 
needing to assume that ^ and $ have the same zeroth moment, that the two solutions coincide when A 
is singular. If A is instead non-singular, the two solutions are typically different. For instance, let 



A^ 



-V6 + 



■ 




' V3 ■ 




1 




2 
V2 




3 




. 3 





(37) 



It is easy to see that J GG* is the unique solution to the Lyapunov equation P — APA^ = BB^ , and in 
this case P ~ I. Thus / G^G* — I, and 5* G §+(T) is compatible with the second-order statistics. Since 
'5KL(^||*i') is a distance measure among spectral densities, our solution coincides with ^' (in fact condition 
(30) holds for A° = 0). When ^1^ = 1, the solution with 5a'lo(^||^') can be expressed in the closed form 
*KLo = {G*B{B*B)-'^B*G)-'^, see pJJ. Substituting the parameters (37) in $klo, 

we obtain 



4'KLo(2) 



42^2 



245z + 434 - 245^-1 -f A2z-^ 



-175z-f 370 - 175Z-1 



(38) 



which is different from the compatible prior. We conclude that our solution preserves the approximation- 
feature (i.e. the solution is as close as possible to ^) also in the case wherein A is invertible. 



5 Case 1 < v < oo 

The corresponding Lagrangian is 
L($,A) = 



,,2 



G$G* 



+ G*AG$ 



/,A 



tr(A). 



(39) 



Similarly to the previous case we can impose that A G Range F. Since 5i,($||5') is strictly convex 
with respect to $ G S+(T), then L{-,A) is strictly convex with respect to $ G S+(T). Accordingly, the 
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corresponding unconstrained minimization problem admits a unique solution given by annihilating the 
first variation of L{-,A) in each direction (5$ G Loo(Tr): 



1^1 



(5L($,A;(5$) = / {-iy<l>-^^- +v + G*AG] 5<i> . (40) 



Note that, -iy$-T7*77 + „ + G* KG e L^{T). Hence, (|40f is zero VJ$ G Loo(T) if and only if 

-^v^-T^^!^ +v + G*KG = Q. (41) 
Therefore, the unique minimum point of L(-, A) has the form 

*(A) := 7 (42) 

(l + iG*AG) 

Since $^i^v]>i7 ^ §^(T), the set of admissible Lagrange multipliers is 

£^ := |a e Q„ I 1 + ^G*AG > on t| n Range T. (43) 

Also in this case $(A) is rational because and G are rational, and v G \ {!}. Moreover, 

deg[$(A)] < deg[*] + 2nv. (44) 
It remains to be shown that the dual functional 

1 



J(A) := -i:($(A),A) = — ^ / * ( 1 + -G*AG) + tr(A) (45) 



admits a minimum point A° over . Accordingly $(A°) is the imique solution to Problem 3.1 
Theorem 5.1 The dual functional J belongs to C^{C^) and it is strictly convex over . 
Proof. The first variation of J(A) in direction 5Ai G Q„ is 



(5J(A;(5Ai) = -y"* (^1 + ^G*AGj G*(5AiG + tr(Mi). (46) 

In order to prove that J(A) G C^(£|^) we have to show that 6J{A;SAi), for any fixed 5Ai, is continuous 
in A. Consider a sequence M„ G Range F such that Af„ and define Qn{z) = (1 + ^G(z)* AG(2;))~^ 
with A'' G Qn- We know that (5a+m„ converges uniformly to Qa, see [El Lemma 5.2]. Thus, applying 
elementwise the bounded convergence theorem, we obtain 

Jirn^ j G*gX+M„G* = j GfQ^G*. (47) 

Accordingly, 5J{A\5A) is continuous, i.e. J belongs to C^{C^). The second variation in direction 
(5Ai,(5A2 G Qn is 

(5V(A;(5Ai,(5A2) = y"* (^1 + Jg*AG^ G*5AiGG*(5A2G (48) 

and the continuity of 5^ J can be established by using the previous argumentation. Finally, it remains to be 
shown that J is strictly convex on the open set i.e. Ha(5A, 5K) :— 5^ J(A; 5A, 5 A) is positive definite 
over . Since the integrand in (48 1 is a nonnegative function when 5Ai = (5A2, we have Ha(i5A, 5 A) > 0. 
If nK{5A,5A) ^ 0, then G*SAG = namely SA G [Range F]^. Since SA G Range F, it follows that 
6 A = 0. Thus, ■Ha('5A, SA) is positive definite. ■ 
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Theorem 5.2 The dual functional J has a unique minimum point in . 

Proof. Since the solution of the dual problem over if it does exist, is unique, we only need to s how 
that J takes a minimum value on . First of all, note that J is continuo us on see Theorem 5.1 
Secondly, we show that tr(A) is bounded from below on . Since Problem 3.1 is feasible, there exists 
$/ e S+(T) such that /G$/G* = /. Thus, 



tr(A) = tr 
Defining 7 = —v J $/, we obtain 



G$/G*A 



= / G*AG$/. (49) 



tr(A) = iy J (^1 + ^G*AG^ $ 



/+7- (50) 

Note that $7 is a coercive spectrum, namely there exists a constant /i > such that $/(e-''') > /i, 
V e-''' € T. Since the integral is a monotonic function, we get 

tr(A) > i/^y 1 + ^G*AG + 7 > 7 (51) 

where we have used the fact that / 1 + iG*AG > when A e . Thirdly, notice that J(0) = ^ / ^f. 
Accordingly, we can restrict the search of a minimum point to the set IC := {A G | J(A) < J(0)}. 
Finally, the existence of the solution to the dual problem follows from the Weierstrass' theorem, since K, 
is a compact set. In order to prove that IC is compact, it is sufficient to show that: 

1. lim J(A) +00; 

2. lim J(A) = +00. 

||A||— >-oo 

Point (1): Function rA{z) := 1 + ^G(z)*AG(z) is rational. Observe that dC^ is the set of A e Range F 
such that rA(e'''') > on T and there exists •& such that rA{e^^) is equal to zero. Thus, for A — > d£^ 
rA{z)~^ has at least one pole tending to the unit circle. Since 1/ — 1 E N+, then r\{z)^~'^ has at least 
one pole (of order greater than or equal to z/ — 1) tending to T. Since ^I* is fixed and coercive, then also 
^(z)rA(z)-'-^'^ has one pole tending to the unit circle. Accordingly, J 'i'r]^'^ — >■ 00 as A — >■ dC^ . In view 



of ( 51 ), we conclude that J(A) = ^ / '^r\ " + tr(A) — > 00 as A — >■ dC 



Point (2): Consider a sequence {A^jfegN G £ , such that 

lim IIAfcll =00. (52) 

Let Al = p^. Since £^ is convex and € if A e then ^A e £^ V ^ G [0, 1]. Therefore A^ e £^ 
for k sufficiently large. Let 77 := liminf tr(A°). In view of (51), 



ti-(A2) = ^tr(A,)>^7^0, (53) 

for fc — 7- 00, so 77 > 0. Thus, there exists a subsequence of {A°} such that the limit of its trace is equal to 
77. Moreover, this subsequence remains on the surface of the unit ball dB = {A = A-^ | ||A|| = l} which 
is compact. Accordingly, it has a subsubsequence {A^.} converging in dB. Let A°° e dB be its limit, 
thus lim tr(A^.) = tr(A°°) = 77. We now prove that A°° e r^. First of all, note that A°° is the hmit of 

z— f 00 ^ 

a sequence in the finite dimensional linear space Range F, hence A°° G Range F. It remains to be shown 
that 1 + j^G*A°°G is positive definite on T. Consider the unnormalized sequence {A^.} G C^: We have 
that 1 + ^G*Afc^G > on T so that + iG*A^.G is also positive definite on T for each i. Taking the 

limit for i — > 00, we get that G*A°°G is positive semidefinite on T so that 1 + -G* A°°G > on T. Hence, 



A°° e C^. Since Problem 3.1 is feasible, there exists $7 S S+(T) such that / = / G<I>7G*, accordingly 



77 = tr(A°°) ^trj G$7G*A°° = J $7G*A°°G > fi J G*A°°G. (54) 
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Moreover, G*A°°G is not identically equal to zero. In fact, if G*A°°G = 0, then A°° e [Range T]^ and 
A°° ^ since it belongs to the surface of the unit ball. This is a contradiction because A°° £ Range F. 
Thus, G*A°°G is not identically zero and 77 > 0. Finally, we have 



lim J(Afe) = hm fl + -G*AfeG) +tr(Afc) 



> lim IIAfcll tr(A5?) > 77 lim ||Afc|| = 00. (55) 

A;— ^00 



Now, it is worth to compare the related work in [20]. Here, the Beta divergence family 

5^(<i>||vl/) = 1 _l_(<i>^ - $vI;/3-i) - ^($^ - ^^), /3 e M \ {0, 1} (56) 

has been considered. Taking the parametrization /? = — ^ + 1 with g N such that 1 < < oo, the 
corresponding optimal form of the spectrum approximation problem is 

$^(A) = (^^-i + ^G*AG^ . (57) 

Thus, not only must ^' be rational, but even '^'^ must be rational in order that <i>^(A) is rational. In this 



situation deg[$^(A)] < j/(deg[^'''] + 2n). We conclude that the solution given by (42| is more appealing 



than the one given by ibll. On the other hand, while (42 1 is only valid for the scalar case, (57) also holds 



for the multichannel case. Finally, note that the two solutions coincides when XE" = 1. 

6 Case u = oo 

In this case we minimize the Kullback-Leibler divergence with respect to the first argument. Thus, the 
corresponding solution follows the principle of minimum discrimination information. Its Lagrangian is 

L($,A) := 5KL($||*)-y"* + <^y"G$G* -/,A^ 

= J $(log($) - log(«')) - $ + G*AG$ - tr(A) 

where A G Range F as in the previous cases. Since L(-,A) is strictly convex over S+(T), its unique 
minimum point is given by annihilating its first directional derivative for each 5$ S Loo(T): 

(5i($, A; (5$) = J [log($) - log(*) + G* AG](5$. (58) 

Since log($) - log(^') + G*AG e Loo(T), the mininmm point for L(-, A) is 

$(A) := ei°s(*)-«*AG (59) 



and the set of admissible Lagrange multipliers is Range F. Note that (59) is an exponential solution 



Thus, it is not rational though is rational. Finally, we show that the dual functional 

J(A) = -L($(A), A) = y *e-^'^^ + tr(A) (60) 



admits a minimum point A° over Range F. Hence, $(A°) is the unique solution to Problem 3.1 
Theorem 6.1 The dual functional J belongs to (Range F) and it is strictly convex over Range F. 
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Proof. The first and the second variation of J(A) in direction 6A E Qn are 

SJ{A;6A) = - j ^e-^'^^G*5AG + tT{6A) 
5^JiA;6A,SA) = / ^e-'^'^^{G*SAGf, (61) 



respectively. Similarly to the previous cases, in order to prove that J(A) G (Range F) we consider a 
sequence M„ e Range T such that M„ -j' and define QAr(z) = e~'^(^)'^'^(^) with N e Q„. By using 
similar argumentations in the proof of Lemma 5.2 in pTS], it is possible to prove that Qa+a/„ converges 
uniformly to Qa- Thus, applying the bounded convergence theorem, we obtain 

lim / G^Qa+m^G* = f G^QaG*. (62) 

n^oo J J 

Accordingly dJ{A;6A), once fixed SA, is continuous in A, i.e. J belongs to C^(Range F). In similar way 
we can establish the continuity of S^J{A;6A,6A). Finally, note that 'Ha{SA,6A) := S^J{A;SA,SA) > 
and 5fe-<=*^<^ e S+(T). Thus, (5V(A; (5A, M) = implies that G*SAG = 0, i.e. 5 A e [Range F]-^. Since 
^A e Range F, we get 5A = 0. We conclude that the Hessian of J is positive definite, and thus J is 
strictly convex over Range F. ■ 



Theorem 6.2 J admits a unique minimum point over Range F. 



Proof. Also in this case it is sufficient to show that J takes a minimum value on Range F. Firstly, note 
that J(0) — ^ ^ . Hence, the search of the minimum point over Range F is equivalent to minimize J over 
the closed set JC :— {A E Range F | J(A) < J(0)}. We want to show that JC is bounded and accordingly 
compact. Then, by Weierstrass' theorem we conclude that J admits a minimum point over /C. To show 
that K, is bounded, we prove that 

lim J(A) = oo. (63) 



l|A||- 



To this aim, note that Problem 
Accordingly, 



3.1 



is feasible. Thus, there exists $/ e S+(T) such that / G^/G* = /. 



tr(A) 



$7G*AG. 



(64) 



Let us consider a sequence {AfcjfcgN, A^ g Range F such that ||Afc|| — >• oo. Consider the sequence A° 



which is contained in the closed ball {A = A-^ 



1}. Let 1^ = liminf tr{Al). Note that jryl < 



l|A».|| ^ 

Consider a subsequence of {A^} such that the limit of its trace is equal to rj. Since this subsequence 
is contained in a compact set, there exists a subsubsequence {A",} having limit Aoo G Range F with 
||Aoo|| = 1- Clearly, tr(Aoo) = ??. Note that Aqo is not equal to the null matrix because ||Aoo|| — 1. 
Moreover G* A^oG ^ because Aoo e Range F. If G* A^oG > 0, then 



r? = tr(Aoo) 



$/G*AooG>M / G*AooG>0, 



(65) 



where we have exploited the fact that $/ e §_|_(T), i.e. there exists > such that $/(e-''') > /i V e^'' G T. 
Accordingly, 

lim J(Afe) > lim ||Afe|| / $/G*A^,G > r/ lim ||Afc|| = c5o. (66) 

In the remaining possible case, there exists ■& such that G* {'d)AryoG{-d) < 0. Thus, G* (t9)A^,G(??) — — oo, 
and accordingly J 'i'e~'^ ^^'^ — > oo, as fc — >■ oo. Moreover the latter dominates the term tr(Afc) ~ 
J ^jG* AkG. Accordingly, we conclude that J(Afc) — > oo as fc — > oo. ■ 



Note that Theorem 6.2 can be proofed by using homotopy-like methods as in [T5] . 
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7 Final remarks and conclusions 



We analyzed a spectrum approximation problem based on a suitable parametrization of the Alpha diver- 
gence family. When the parameter e N is such that 1 < u < oo^ the corresponding family of solutions 
is rational with an upper bound on the degree equal to deg[5'] + 2nv. Moreover, the solution having the 
smallest upper bound is given by minimizing the Kullback-Leibler divergence with respect to the second 
argument (case 11 = 1). Such solution also generalizes the approximation presented in [TB]. For v — oo we 
obtain the minimum discrimination information solution (MinxEnt) which is non-rational. Furthermore, 
there exists a "connection" among these rational solutions and the MinxEnt solution. 



Proposition 7.1 Let <^i,{K) denote the optimal form (4-2) and $oo(A) denote the optimal form ^ 
Then <i>iy(A) $00 (A) as v 00. 



Proof. We want to show that ^-(1 + ^G* KG)-" ^-e-^'^^ as v 00. Since lim (1 + ^G*kGY 

„G*AG 



we get 



lim $,y(A) = lim 



i/^oo (1 + ^G*kGY 



(67) 



Let A° and A^ be the optimal Lagrange multipliers of ( 45 1 and ( 60 1 , respectively. By Proposition 7. 1 
we cannot conclude that <f>^(A°) — )■ $00 (A^) as z/ — 00. However, simulations suggest this conjecture 
is true. To illustrate this fact, we take into account the ARMA process considered in Section VIIB] 
with spectral density 



We choose as filters bank 



z5 - 0.5z4 + 0.42z3 - 0.602z2 4- 0.0425z - 0.1192' 



G(z) 



and the corresponding output covariance is 



E = / GilG* 
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The a priori information on the ARMA process is given by the prior 

^ ' z-0.82 



(68) 



(69) 



(70) 



(71) 



In FigurejTJthe approximations of ^' compatible with S for i^ = l, j/ = 2, i^ = 4 and h' = 00 are depicted. 
It is clear that increasing 1/ the corresponding solution approaches the MinxEnt solution, and for v = 4 it 
is pretty similar to the MinxEnt one. We have conducted other numerical experiments and we observed 
the same behaviour as v increases. 

We conclude that the Alpha divergence family yields a concrete improvement in the spectrum approx- 
imation problems both for rationality property of the family (when < 00), under the mild assumption 
that ^' is rational, and for the closeness of the solutions to the MinxEnt solution for ly sufficiently large. 
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Figure 1: Approximations of ^ with different values of v. 
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